Automated occupation coding with hierarchical features: a data-centric approach to classification with pre-trained language models

Abstract

Occupation coding is the classification of information on occupation that is collected in the context of demographic variables. Occupation coding is an important, but a tedious task for researchers in social science and official statistics that calls for automation. Due to the complexity of the task, currently, researchers carry out hand-coding or computer-assisted coding. However, we argue that, with the rise of transformer-based language models, hand-coding can be displaced by models, such as BERT or GPT3. Hence, we compare these models with state-of-the-art encoding approaches, showing that language models have a clear advantage in Cohen’s kappa compared to related approaches, but also allow for flexible fine-grained coding of single digits. Taking into consideration the hierarchical structure of the occupational group, we also develop an approach that achieves better performance for the classification of different single digit combinations.

Opportunities and Challenges in Data-Centric AI

“…They highlight the importance of context-based data preparation and demonstrate that improving data quality has a significant impact on prediction accuracy. Safikhani et al [111] argue that transformer-based language models can enhance automated occupation coding. By fine-tuning BERT and GPT3 on pre-labeled data and incorporating job titles and task descriptions, they achieve a 15.72 percentage point performance increase compared to existing methods.…”

Potential Impact of Data-Centric AI on Society

“…Recently, several research articles have proposed DCAI methods in different domains. These include data-centric defense (DCD) for mitigating model inversion attacks [10], crop disease identification in agriculture [11], unsupervised anomaly detection in industrial production [12], transformer-based language models (bidirectional encoder representations from transformers (BERT) and GPT-3 for automated occupation coding [13], enhancing deep neural network (DNN) model robustness [14], a comprehensive community library for biomedical natural language processing (NLP) data sets [15], automatic surgical phase estimation [16], rapid nanoparticle energy prediction [17], identifying incongruous regions in data [18], a multidomain benchmark for Table 1. Brief summary of related works in DCAI.…”