On personalizing Web content through reinforcement learning

Ferretti, Stefano; Mirri, Silvia; Prandi, Catia; Salomoni, Paola

doi:10.1007/s10209-016-0463-2

Cited by 20 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For several decades, both academia and commerce have sought to develop tailored products and services at low cost in various application domains. These reach far and wide, including medicine [5,71,77,84,178,179], human-computer interaction [61,100,118], product, news, music and video recommendations [163,164,217] and even manufacturing [39,154]. When products and services are adapted to individual tastes, they become more appealing, desirable, informative, e.g.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning for personalization: A systematic literature review

Hengst

Grua

Hassouni

et al. 2020

View full text Add to dashboard Cite

The major application areas of reinforcement learning (RL) have traditionally been game playing and continuous control. In recent years, however, RL has been increasingly applied in systems that interact with humans. RL can personalize digital systems to make them more relevant to individual users. Challenges in personalization settings may be different from challenges found in traditional application areas of RL. An overview of work that uses RL for personalization, however, is lacking. In this work, we introduce a framework of personalization settings and use it in a systematic literature review. Besides setting, we review solutions and evaluation strategies. Results show that RL has been increasingly applied to personalization problems and realistic evaluations have become more prevalent. RL has become sufficiently robust to apply in contexts that involve humans and the field as a whole is growing. However, it seems not to be maturing: the ratios of studies that include a comparison or a realistic evaluation are not showing upward trends and the vast majority of algorithms are used only once. This review can be used to find related work across domains, provides insights into the state of the field and identifies opportunities for future work.

show abstract

Section: Introductionmentioning

confidence: 99%

Reinforcement learning for personalization: A systematic literature review

Hengst

Grua

Hassouni

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Although there are many general guidelines to ensure web accessibility, it is almost impossible to provide all needs, demands and preferences of heterogeneous user groups at the same time. Therefore, some experts recommend using a customisable web environment [9,16] with the possibility of website transformation [17,18] in accordance with users’ individual needs and preferences. This means that users have the opportunity to customisable the website by themselves in accordance with their individual needs and preferences.…”

Section: Discussionmentioning

confidence: 99%

“…Therefore, recommendations for designing universal websites can be challenging because a single design may not work for everyone due to the different needs and preferences of individuals with dyslexia [15]. Therefore, some authors recommend using a customisable web environment [9,16] with the possibility of website transformation [17,18] in accordance with users’ individual needs and preferences.…”

Section: Introductionmentioning

confidence: 99%

“…Personalisation and customisation are well-known strategies and they have also been implemented in order to improve access to the Web and to improve the user experience [17,19]. The existing literature reports some solutions for the personalisation and customisation of websites for people with dyslexia, such as web-based services [20], a plugin for browsers [21] and transcode techniques [17]. This research focused on the customisation of website in accordance with users’ individual needs and preferences.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Empirical Insights of Individual Website Adjustments for People with Dyslexia

Kous

Polančič

2019

Sensors

View full text Add to dashboard Cite

The existing body of knowledge reveals that customisable websites may lead to an increase in accessibility and usability for people with disabilities. In this way, the main goal of this research was to investigate how people with dyslexia respond to a customised version of a website in terms of its effectiveness, efficiency, satisfaction and suitability when compared to the default version of the website. The customisation of the investigated website was enabled with the aid of integrated assistive technology that offers people with dyslexia the opportunity to adjust a website themselves in accordance with their individual needs, demands and preferences. They can do this by changing the parameters, such as font size, font type and contrast between the background and text. The answers to the research questions were obtained with complementary research methods and techniques, including formal usability testing, thinking aloud protocol, log analyses, questionnaires and interviews. The empirical results show that participants experienced more issues when interacting with the default website, and they enjoyed more benefits when using the customised website. Too much information on the screen, not enough graphic elements, issues with visual appearance and inappropriately presented information were identified as the most common issues when interacting with the default website. When using the customised website, all participants agreed on a better user experience and, as the majority of them reported, this was due to appropriate contrast and font size. Additionally, the majority of participants also expressed desire to use the individual website adjustments regularly in the future. The conclusions of this investigation are that the individual website adjustments used in this research can not only help to minimise issues, but also eliminate challenges that people with dyslexia have when interacting with a website. Therefore, the primary contributions of this research are the empirical insights of interaction with both the default and customised version of the website for people with dyslexia. Furthermore, this research also has three secondary contributions: (1) detailed presentation and application the general usability evaluation procedure to a specific target group (people with dyslexia); (2) recommendations to adapt the usability evaluation methods for people with dyslexia; and (3) the usage of quantitative measurement instruments for the evaluation of a website’s usability and suitability for people with dyslexia.

show abstract

“…For several decades, both academia and commerce have sought to develop tailored products and services at low cost in various application domains. These reach far and wide, including medicine [12,123], human-computer interaction [102,215], product, news, music and video recommendations [291,293,381] and even manufacturing [67,272]. When products and services are adapted to individual tastes, they become more appealing, desirable, informative, e.g.…”

Section: Introductionmentioning

confidence: 99%

Learning to Behave

den Hengst

View full text Add to dashboard Cite

Reinforcement learning (RL) has recently attracted significant attention with applications such as improving microchip designs, predicting the behaviour of protein structures and beating humanity’s best in the games of go, chess and Starcraft-II. These impressive and inspring successes show how RL can improve our lives, however, they have so far been seen mostly in settings that involve humans to a very limited extend. This thesis looks into the usage of RL in human contexts. First, we provide a systematic literature review of the usage of RL for personalisation, i.e. the adaptation of systems to individuals. Next, we show how RL can be used to personalise a conversational recommender system and find that it outperforms existing approaches, including a gold-standard and task-specific solutions in a simulation-based study. Since simulators may not be available for all conversational systems that could benefit from personalisation, we next look into the collection of user satisfaction ratings for dialogue data. We consolidate best practices in a UI for user satisfaction annotation and show that high-quality ratings can be obtained. Next, we look into the usage of RL for strategic workforce planning. Here, we find that RL is robust to the uncertainties that are an inherent part of this problem and that RL enables the specification of goals intuitive to domain experts. Having looked into these use-cases, we then turn toward the inclusion of safety constraints in RL. We propose how safety constraints from a medical guideline can be taken into account in an observational study on the optimisation of ventilator settings for ICU patients. Next, we look into safety constraints that contain a temporal component, we find that these may make the learning problem infeasible and propose a solution based on reward shaping to address this issue. Finally, we propose how RL can benefit from instructions that break a full task into smaller pieces based on the option framework and propose an approach for learning reusable behaviours from instructions to greatly reduce data requirements.

show abstract

On personalizing Web content through reinforcement learning

Cited by 20 publications

References 27 publications

Reinforcement learning for personalization: A systematic literature review

Reinforcement learning for personalization: A systematic literature review

Empirical Insights of Individual Website Adjustments for People with Dyslexia

Learning to Behave

Contact Info

Product

Resources

About