Amherst685 at SemEval-2021 Task 7: Joint Modeling of Classification and Regression for Humor and Offense

Abstract

This paper describes our submission to the SemEval'21: Task 7-HaHackathon: Detecting and Rating Humor and Offense. In this challenge, we explore intermediate finetuning, backtranslation augmentation, multitask learning, and ensembling of different language models. Curiously, intermediate finetuning and backtranslation do not improve performance, while multitask learning and ensembling do improve performance. We explore why intermediate finetuning and backtranslation do not provide the same benefit as on other natural language processing tasks and offer insight into the errors that our model makes. Our best performing system ranks 7th on Task 1b with an RMSE of 0.5339.