Online learning to diversify from implicit feedback

Raman, Karthik; Shivaswamy, Pannaga; Joachims, Thorsten

doi:10.1145/2339530.2339642

Cited by 52 publications

(47 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The first source is likely to increase and other data sources are fined due to fewer numbers of user feedback. Raman et al have proposed online learning-based ranking algorithm to create a balance between relevance and diversity [32]. At each stage, a user is provided with the results of ranking algorithms.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Web pages ranking algorithm based on reinforcement learning and user feedback

Derhami

Paksima

Khajeh

2015

JAIDM

View full text Add to dashboard Cite

The main challenge of a search engine is ranking web documents to provide the best response to a user`s query. Despite the huge number of the extracted results for user`s query, only a small number of the first results are examined by users; therefore, the insertion of the related results in the first ranks is of great importance. In this paper, a ranking algorithm based on the reinforcement learning and user`s feedback called RL3F are considered. In the proposed algorithm, the ranking system has been considered to be the agent of learning system and selecting documents to display to the user is as the agents' action. The reinforcement signal in the system is calculated according to a user`s clicks on documents. Action-value values of the proposed algorithm are computed for each feature. In each learning cycle, the documents are sorted out for the next query, and according to the document in the ranked list, documents are selected at random to show the user. Learning process continues until the training is completed. LETOR3 benchmark is used to evaluate the proposed method. Evaluation results indicated that the proposed method is more effective than other methods mentioned for comparison in this paper. The superiority of the proposed algorithm is using several features of document and user`s feedback simultaneously.

show abstract

Section: Related Workmentioning

confidence: 99%

“…At each stage, a user is provided with the results of ranking algorithms. Theoretically, the efficiency of the algorithm has increased and the algorithm is resistant to noise [32]. DBGD ranking algorithm is online and based on reinforcement learning [33,34].…”

Section: Related Workmentioning

confidence: 99%

Web pages ranking algorithm based on reinforcement learning and user feedback

Derhami

Paksima

Khajeh

2015

JAIDM

View full text Add to dashboard Cite

show abstract

“…Recently, coactive learning [14] has been applied to the problem of intrinsic diversity. As opposed to our problem (i.e., extrinsic diversity [1]) intrinsic diversity is diversity required by a single individual among their various different interests.…”

Section: Related Workmentioning

confidence: 99%

“…As done in previous work [14], we assume that the utility functions U i (x t , y t ) is linear in its parameters v i ∈ R m .…”

Section: Submodular Utility Modelmentioning

confidence: 99%

“…Note that this setup is fundamentally different from previous coactive learning problems [13][14][15], which assume that user actions always come from the same utility function. After characterizing the informativeness and noisiness of the implicit feedback, we give theoretical results bounding the regret of our algorithms in terms of the social utility.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Socially Optimal Information Systems from Egoistic Users

Raman

Joachims

2013

Advanced Information Systems Engineering

Self Cite

View full text Add to dashboard Cite

Abstract. Many information systems aim to present results that maximize the collective satisfaction of the user population. The product search of an online store, for example, needs to present an appropriately diverse set of products to best satisfy the different tastes and needs of its user population. To address this problem, we propose two algorithms that can exploit observable user actions (e.g. clicks) to learn how to compose diverse sets (and rankings) that optimize expected utility over a distribution of utility functions. A key challenge is that individual users evaluate and act according to their own utility function, but that the system aims to optimize collective satisfaction. We characterize the behavior of our algorithms by providing upper bounds on the social regret for a class of submodular utility functions in the coactive learning model. Furthermore, we empirically demonstrate the efficacy and robustness of the proposed algorithms for the problem of search result diversification.

show abstract

Automating Layout Synthesis with Constructive Preference Elicitation

Erculiani

Dragone

Teso

et al. 2019

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Layout synthesis refers to the problem of arranging objects subject to design preferences and structural constraints. Applications include furniture arrangement, space partitioning (e.g. subdividing a house into rooms), urban planning, and other design tasks. Computeraided support systems are essential tools for architects and designers to produce custom, functional layouts. Existing systems, however, do not learn the designer's preferences, and therefore fail to generalize across sessions or instances. We propose addressing layout synthesis by casting it as a constructive preference elicitation task. Our solution employs a coactive interaction protocol, whereby the system and the designer interact by mutually improving each other's proposals. The system iteratively recommends layouts to the user, and learns the user's preferences by observing her improvements to the recommendations. We apply our system to two design tasks, furniture arrangement and space partitioning, and report promising quantitative and qualitative results on both.

show abstract

Online learning to diversify from implicit feedback

Cited by 52 publications

References 14 publications

Web pages ranking algorithm based on reinforcement learning and user feedback

Web pages ranking algorithm based on reinforcement learning and user feedback

Learning Socially Optimal Information Systems from Egoistic Users

Automating Layout Synthesis with Constructive Preference Elicitation

Contact Info

Product

Resources

About