Annotated Span Normalization as a Sequence Labelling Task

Abstract

In this paper, we describe a way to perform span normalization as a sequence labelling task. Our model predicts the modifications that should be applied to the span tokens to normalize them. This prediction is performed via se quence labelling, which means that each token is normalized independently. Despite the simplicity of the approach, we show that it can lead to the stateoftheart results. We compare different pretraining schemas in application to this task. We show that the best quality can be achieved when the normalizer is trained on top of a BERTbased morphosyntactic parser's representations. Moreover, we propose some additional features useful in the task and prove that auxiliary morphosyntactic losses can help the model. Furthermore, we show that the model compares favourably with other contestant models of the RuNormAS competition.