BERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter

Abstract

This paper describes Kata.ai's submission for the Social Media Mining for Health (SMM4H) 2021 shared task. We participated in three tasks: classifying adverse drug effect, COVID-19 self-report, and COVID-19 symptoms. Our system is based on BERT model pre-trained on the domain-specific text. In addition, we perform data cleaning and augmentation, as well as hyperparameter optimization and model ensemble to further boost the BERT performance. We achieved the first rank in both classifying adverse drug effects and COVID-19 selfreport tasks.

The unreasonable effectiveness of machine learning in Moldavian versus Romanian dialect identification

“…67,68 As one of the most fundamental tasks in natural language processing (NLP), text classification has been widely studied. 69 Examples of setups and applications are (but not limited to) social media, 70 healthcare, [71][72][73] information retrieval, 74 sentiment analysis, [75][76][77][78][79] content-based recommender systems, 80 document summarization, 81,82 various business and marketing applications, [83][84][85] and legal document categorization. 86 A variety of languages were targeted over time for the popular text classification task, including well-studied languages, such as Arabic, 87,88 Turkish, 83,[89][90][91] French, 71,92 Spanish, 72 and Indian, 93 as well as underresourced languages, such as Romanian.…”