Real-time recommendation of diverse related articles

Abstract

News articles typically drive a lot of traffic in the form of comments posted by users on a news site. Such usergenerated content tends to carry additional information such as entities and sentiment. In general, when articles are recommended to users, only popularity (e.g., most shared and most commented), recency, and sometimes (manual) editors' picks (based on daily hot topics), are considered. We formalize a novel recommendation problem where the goal is to find the closest most diverse articles to the one the user is currently browsing. Our diversity measure incorporates entities and sentiment extracted from comments. Given the realtime nature of our recommendations, we explore the applicability of nearest neighbor algorithms to solve the problem. Our user study on real opinion articles from aljazeera.net and reuters.com validates the use of entities and sentiment extracted from articles and their comments to achieve news diversity when compared to content-based diversity. Finally, our performance experiments show the real-time feasibility of our solution.

Search Result Diversification in Short Text Streams

“…He et al [18] propose a result diversification framework based on query-specific clustering and cluster ranking, in which diversification is restricted to documents belonging to clusters that potentially contain a high percentage of relevant documents. More recent implicit work includes set-based recommendation of diverse articles [1], term-level diversification [14], diversified data fusion [26], and neural-network-based diversification model [46]. Abbar et al [1] address the problem of providing diverse news recommendations related to an input article by leveraging user-generated data to refine lists of related articles.…”

“…More recent implicit work includes set-based recommendation of diverse articles [1], term-level diversification [14], diversified data fusion [26], and neural-network-based diversification model [46]. Abbar et al [1] address the problem of providing diverse news recommendations related to an input article by leveraging user-generated data to refine lists of related articles. They explore different diversity distances that rely on the content of user comments on articles such as sentiments and entities.…”

“…We work with publicly downloadable posts that were a 1% sample from Twitter as a short text stream. 1 The tweets were posted between February 1, 2015, and April 30, 2015, covering a period of about 90 days. Most tweets are written in English.…”

“…We evaluate our proposed algorithm for search result diversification in short text streams on a large dataset consisting of a three-month sample of Twitter and compare it to three types of search result diversification methods: (1) algorithms that do not consider data streams, such as xQuAD; (2) streaming diversification methods that work with data streams but that have been developed for long documents, such as in Refs. [7,33]; and (3) algorithms that combine existing dynamic topic models with effective diversification retrieval models that have not been designed to work with text streams.…”

http://www.sersc.org/journals/IJSEIA/vol11_no7_2017/1.pdf

“…Also, recommendation systems and techniques are being used for many years, most of these techniques never concerned about the performance with high speed data streams. Studies have been interested in making batch processing that cope with data streams, incremental batch approaches, or in comparison of recommendation algorithms and their limitations using batch and online processing [7].…”