Learning from human feedback has shown success in aligning large, pretrained models with human values. Prior works have mostly focused on learning from high-level labels, such as preferences between pairs of model outputs. On the other hand, many domains could benefit from more involved, detailed feedback, such as revisions, explanations, and reasoning of human users. Our work proposes using nuanced feedback through the form of human revisions for stronger alignment. In this paper, we ask expert designers to fix layouts generated from a generative layout model that is pretrained on a large-scale dataset of mobile screens. Then, we train a reward model based on how human designers revise these generated layouts. With the learned reward model, we optimize our model with reinforcement learning from human feedback (RLHF). Our method, Revision-Aware Reward Models (RARE), allows a generative text-to-layout model to produce more modern, designer-aligned layouts, showing the potential for utilizing human revisions and stronger forms of feedback in improving generative models.

In order to further improve students' English application abilities, the establishment of follow-up college English courses is urgent for TCM universities. The follow-up college English curriculum should aim on improving students' specialized English level as well as cultivating their humanistic quality. The authors suggested multi-module and multi-level courses should be considered in the construction of this system. With its scientific nature, individualized course management, specialized teaching materials, the construction of follow-up college English curriculum will not only integrate the students' specialized English skill with cultural literacy but also provide a direction for the development of English teachers in TCM colleges and universities.

No abstract