An Experimental Study of Bias in Platform Worker Ratings: The Role of Performance Quality and Gender

Jahanbakhsh, Farnaz; Cranshaw, Justin; Counts, Scott; Lasecki, Walter S.; Inkpen, Kori

doi:10.1145/3313831.3376860

Cited by 15 publications

(10 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In both studies, significant differences across experimental conditions were observed for female users but not males. This is consistent with other recent studies [36,24], where female and male Mechanical Turk Workers reacted differently to issues related to race and gender. This is an important takeaway for researchers who use Amazon Mechanical Turk.…”

Section: Female Vs Male Workers On Amazon Mechanical Turksupporting

confidence: 93%

“…Developing solutions to mitigate bias is currently a major challenge in AI [6,1,38,35,4]. Implicit gender and racial biases in technology is being studied by both AI and HCI researchers [34,38,35,24,36], and especially in the context of recidivism following ProPublica's report that COMPAS, a nationwide criminal risk assessment tool, contained racial disparities in its predictions [1,46,17,11,8,43,42].…”

Section: Related Workmentioning

confidence: 99%

“…Prior research has shown that the gender of an Amazon Mechanical Turk worker can have a significant effect on results, particularly when gender or race is involved in the task [24,36]. We therefore investigated whether users' self-identified gender impacted recidivism predictions for our experimental conditions.…”

Section: User Gendermentioning

confidence: 99%

See 2 more Smart Citations

Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism

Mallari

Inkpen

Johns

et al. 2020

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

Understanding how racial information impacts human decision making in online systems is critical in today's world. Prior work revealed that race information of criminal defendants, when presented as a text field, had no significant impact on users' judgements of recidivism [13]. We replicated and extended this work to explore how and when race information influences users' judgements, with respect to the saliency of presentation. Our results showed that adding photos to the race labels had a significant impact on recidivism predictions for users who identified as female, but not for those who identified as male. The race of the defendant also impacted these results, with black defendants being less likely to be predicted to recidivate compared to white defendants. These results have strong implications for how systemdesigners choose to display race information, and cautions researchers to be aware of gender and race effects when using Amazon Mechanical Turk workers.

show abstract

Section: Female Vs Male Workers On Amazon Mechanical Turksupporting

confidence: 93%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism

Mallari

Inkpen

Johns

et al. 2020

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Consequently, algorithms aimed at supporting decision processes, especially in high-risk contexts such as criminal justice, cannot be developed without taking into account the influences that institutional, behavioural, and social aspects have on the decisions [58]. Furthermore, human factors such as biases, preferences and deviating objectives can also influence the effectiveness of algorithm-supported decision making [40,48].…”

Section: Risk Assessment Instruments (Rai) For Criminal Recidivismmentioning

confidence: 99%

A Comparative User Study of Human Predictions in Algorithm-Supported Recidivism Risk Assessment

Portela¹,

Castillo²,

Tolan³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we study the effects of using an algorithm-based risk assessment instrument to support the prediction of risk of criminal recidivism. The instrument we use in our experiments is a machine learning version of RiskEval (name changed for double-blind review), which is the main risk assessment instrument used by the Justice Department of Country (omitted for double-blind review).The task is to predict whether a person who has been released from prison will commit a new crime, leading to re-incarceration, within the next two years. We measure, among other variables, the accuracy of human predictions with and without algorithmic support. This user study is done with (1) general participants from diverse backgrounds recruited through a crowdsourcing platform,(2) targeted participants who are students and practitioners of data science, criminology, or social work and professionals who work with RiskEval. Among other findings, we observe that algorithmic support systematically leads to more accurate predictions from all participants, but that statistically significant gains are only seen in the performance of targeted participants with respect to that of crowdsourced participants. We also run focus groups with participants of the targeted study to interpret the quantitative results, including people who use RiskEval in a professional capacity. Among other comments, professional participants indicate that they would not foresee using a fully-automated system in criminal risk assessment, but do consider it valuable for training, standardization, and to fine-tune or double-check their predictions on particularly difficult cases.

show abstract

“…More recently, Jahanbakhsh et al [9] investigated the interaction of gender and performance on worker ratings in a simulated teamwork task on Amazon Mechanical Turk. They found that when male and female coworkers were equally low performing, the female worker received worse evaluations.…”

Section: Empirical Evidence Of Gender Biasmentioning

confidence: 99%

Does Fair Ranking Improve Minority Outcomes? Understanding the Interplay of Human and Algorithmic Biases in Online Hiring

Sühr¹,

Hilgard²,

Lakkaraju³

2020

Preprint

View full text Add to dashboard Cite

Ranking algorithms are being widely employed in various online hiring platforms including LinkedIn, TaskRabbit, and Fiverr. Since these platforms impact the livelihood of millions of people, it is important to ensure that the underlying algorithms are not adversely affecting minority groups. However, prior research has demonstrated that ranking algorithms employed by these platforms are prone to a variety of undesirable biases. To address this problem, fair ranking algorithms (e.g., Det-Greedy) which increase exposure of underrepresented candidates have been proposed in recent literature. However, there is little to no work that explores if these proposed fair ranking algorithms actually improve real world outcomes (e.g., hiring decisions) for minority groups. Furthermore, there is no clear understanding as to how other factors (e.g., job context, inherent biases of the employers) play a role in impacting the real world outcomes of minority groups.In this work, we study how gender biases manifest in online hiring platforms and how they impact real world hiring decisions. More specifically, we analyze various sources of gender biases including the nature of the ranking algorithm, the job context, and inherent biases of employers, and establish how these factors interact and affect real world hiring decisions. To this end, we experiment with three different ranking algorithms on three different job contexts using real world data from TaskRabbit. We simulate the hiring scenarios on TaskRabbit by carrying out a large-scale user study with Amazon Mechanical Turk. We then leverage the responses from this study to understand the effect of each of the aforementioned factors. Our results demonstrate that fair ranking algorithms can be an effective tool at increasing hiring of underrepresented gender candidates but induces inconsistent outcomes across candidate features and job contexts.

show abstract

An Experimental Study of Bias in Platform Worker Ratings: The Role of Performance Quality and Gender

Cited by 15 publications

References 42 publications

Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism

Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism

A Comparative User Study of Human Predictions in Algorithm-Supported Recidivism Risk Assessment

Does Fair Ranking Improve Minority Outcomes? Understanding the Interplay of Human and Algorithmic Biases in Online Hiring

Contact Info

Product

Resources

About