Context-Aware Discrimination Detection in Job Vacancies using Computational Language Models

Abstract

Discriminatory job vacancies are a recognized problem with strong impact on inclusiveness and belongingness. It is disapproved worldwide, but remains persistent. Discrimination in job vacancies can be explicit by directly referring to demographic memberships of candidates. On the other hand, more implicit forms of discrimination are also present that may not always be illegal but still influence the diversity of applicants. Although there is a shift towards using implicit forms, explicit written discrimination is still present in numerous job vacancies, as was recently observed for age discrimination in the Netherlands. The studies demonstrated that the lower bound for age discrimination in job vacancies was approximated between 0.16% and 0.24%, while further in-depth analyses showed actual numbers rise above the conservative lower bound. Current efforts for the detection of explicit discrimination concern the identification of job vacancies containing potentially discriminating terms such as "young" or "male". In this way, the automatic detection of potentially discriminatory job vacancies is, however, very inefficient due to the consequent low precision: for instance "we are a young company" or "working with mostly male patients" are phrases that contain explicit terms, while the context shows that these do not reflect discriminatory content.In this paper, we show how state-of-the-art machine learning based computational language models can raise precision in the detection of explicit discrimination by identifying when the potentially discriminating terms are used in a discriminatory context, indicating that the sentence is indeed discriminating. We focus on gender discrimination, which indeed suffers from low precision when filtering explicit terms. First, in collaboration with oversight authorities we created a data set for gender discrimination in job vacancies.Second, we investigated a variety of machine learning based computational language models for discriminatory context detection.Third, we explored and evaluated the capability of these models to detect unforeseen discriminating terms in context. The results show that machine learning based methods can detect explicit gender discrimination with high precision and these state-of-the-art natural * Both authors contributed equally to this research.